In computing environments, a user interface (UI) typically allows a user to interact with objects displayed on a display device by using an input device. For example, a user may use a mouse to direct selection indicia, such as a pointer, to an object on a monitor screen, and then may “click” on the object to select the object or to perform a function on the object. Such a function is typically defined and controlled by the software that is generating the particular UI, or by software that is running transparently to generate the functionality while other software generates the UI. Sometimes a function that may be performed by the user is defined by the area that is selected, or by the area over which the pointer is placed prior to selection. In other instances, the functions that may be performed are contextual, where a function is made available to a user based on what task is being performed, or by the component of the software that is currently being used. In still other instances, a combination of context and user selection determines an available function.
A computer user may use the same pointer to perform a multitude of tasks. For example, a pointer may enable a default function, such as the ability to select objects on a display device, but when the pointer is placed on the edge of an object it may offer a different function, such as a resizing function. When the user moves the pointer off the edge, the pointer may then revert to its default function. As a more detailed example, a user may direct a selection pointer to an object and then may select the object. In many computer systems, such a selection may be accomplished by moving a mouse to position the pointer over the desired object, and then by pressing a button on the mouse (“mouse down”) to select the object. Now that the object has been selected, the software may associate this UI event—where the user has placed the pointer over an object and then pressed a button—with a desire to move the object to another location. Such an association is typically referred to as a component—where a UI event defines a function that the software will perform. Accordingly, the software may enable a relocation function, where the user may move the mouse while holding down the button to move the object to another location on the display device. Upon reaching the desired location, the user may release the button (“mouse up”) to fix the object to its new location. Upon completing the movement, the pointer may revert to being a selection pointer, or it may allow the user to perform another function.
As discussed above, the functions performed by the software are typically activated by events initiated by components, such as a component associated with the aforementioned combination of a mouse movement and button click. Correspondingly, for a given component, user actions typically have a fixed meaning. Therefore, a combination of a component and a UI event can be associated with a “handler,” which is a piece of software code activated by the event. The handler contains computer-readable instructions enabling the computer to carry out the necessary functionality.
In addition to a UI event, and as discussed briefly above, the context in which the UI event occurs may affect which software function is performed, and which handler is activated. For example, in a design environment, such as an editing mode for permitting user interaction with an electronic document, the meaning of a given UI event may vary greatly. The meaning may depend on a particular editing mode in which the software is currently operating, the editing operation currently being performed, the software tool that is currently active, and the like. For example, in a software application having a graphical image of a button on a display device, such as a “print” button in a word processor, the UI event of a mouse click on the button could mean different things depending on the context in which the UI event takes place. For example, it could mean the selection of the button to cause a document to print, the start of a movement of the selected button to another location on the display device, or the activation of text editing in the button's label. In each case, the software may be operating in a different editing mode, such as a general document editing mode, a button relocation mode or a button editing mode, respectively.
Because of the variety of editing operations that may be performed for a given UI event, therefore, UI event processing in an editing environment cannot be tied to particular components. Instead, UI event processing should be handled by a special editing framework. In conventional systems, such a framework involves a systematic means for keeping track of the particular state in which a program is operating. Using the object relocation example given above, a state machine or the like typically performs such a function.
For example, the software in the above example may be operating in a general editing state, in which the default function for the pointer is as a selection pointer. While the pointer is positioned over empty space, or where the pointer is not positioned over any object with which the pointer can interact, a mouse down will not have any effect. Once the pointer is positioned over an object with which it can interact, the state changes to a second state where a mouse down will select the object. However, the user may then reposition the mouse over empty space where the state will revert to the general editing state. Alternatively, the user may mouse down on the object, whereby the state again changes to a third state where a movement of the mouse will move the object. Once a mouse up occurs, the state changes again. In this case, the pointer is likely still over the object, so the state will likely revert to the second state. Another possibility is that the state will change to a fourth state, where another function is available to the user. In addition, given the same set of UI events, if the user is in a different editing mode, any or all of the states may be different, because a different editing mode may, as discussed above, have different functionality for a given UI event.
A state machine in such a conventional system keeps track of all the possible previous and next states in which the software may operate. As in the above example, when in the general editing state, the state machine would permit a user to enter into the second state when the user positions the pointer over an object with which it can interact. Once in the second state, the state machine would permit the user to revert to the previous general editing state, which could occur if the user repositioned the pointer over empty space. Alternatively, the state machine could permit the user to enter the third state, which could occur if the user moused down on the object. There may be a plurality of states into which the user may enter at any point. For example, while in the second state, the user may be able to enter any one of several states—such as a third, fourth, fifth or sixth state—depending on the UI event initiated by the user.
In a conventional system, the state machine may also be comprised of several modules. For example, a first module may keep track of the states as mentioned above. A second module may invoke the necessary handler to perform the function requested by the UI event, and a third module may handle communications and other functions between the first and second module. In the example discussed above, for example, when the user mouses down on the object while in the second state, the first module would note that the user is now in the third state. The second module would invoke the necessary handler(s) to perform the move function, such as a graphics module. The third module would carry out communications between the first and second modules, such as signaling the first module when the handler invoked by the second module has completed its task.
A particularly problematic arrangement occurs in editing environments that involve different types of editable objects, such as mixed text, graphics and installed objects, which results in heterogeneous selection events. Managing a user selection in such an environment is a challenge both in terms of state management and control of multiple software components, each of which are invoked by the same input events. For example, a mouse down may have a certain effect when the selected object is text, and a different effect when the selected object is graphics, a hyperlink or the like, even though the actual UI event—a mouse down—is identical in either case.
As may be appreciated, therefore, any software having a rich set of functionality will have a large and complex arrangement of possible states. In addition, the states must be accounted for with perfect accuracy, otherwise inconsistent results or program failure may occur. For example, if the states are not kept perfectly consistent, the same UI event in the same editing mode may yield a different software operation for a given UI event, or may cause the program to crash by causing the state machine to enter into an inconsistent or unplanned-for state. Using the module system discussed above to provide an example of an inconsistent state, the third module may report to the first module that a certain handler has completed its task as invoked by the second module, but the identity of the handler may not correspond to a proper handler for the state in which the first module is currently operating. In such a case, there may be no instructions for the first module to determine the state to which it should proceed.
In many applications, a user such as a programmer or system administrator may wish to customize the software to add functionality to a program that was not originally part of such program. For example, a user with specific requirements may wish to provide an added or different function from the default function when a pointer is moved to an object. Accommodating such a customization adds a requirement to the software to enable UI event handling to be customizable, so that custom tools can be integrated into the software.
A shortcoming of conventional software is that incorporating such added functionality into the software can be extremely difficult and complex. For example, in the software discussed above, a user wishing to modify the software would need perfect knowledge of each state used by the software, so a new function could be added without causing an illegal function or software crash. If the software has rich functionality, as discussed above, the complexity of the accounting for each of the existing states may cause such a modification to be unduly difficult. In fact, the task is so complicated in conventional software that in most situations, a programmer wishing to customize such conventional software will simply replace the entire UI event handling system rather than attempting to incorporate a new function. Such a replacement is an unnecessarily drastic procedure, particularly when the amount of desired customization is relatively small.
What is needed is a method for providing a framework for extensible UI event handling in a software application. More particularly, what is needed is a method for enabling a UI event handling system to be customized with custom tools, custom types of editable objects, and the like. Furthermore, what is needed is such a method that also adds a mechanism for modification of existing editing tools without the need to completely replace the entire UI event handling system. Even more particularly, what is needed is such a method that further takes advantage of customizable UI event handling to provide an extensible selection mode, where custom component types may participate in selection and editing of an electronic document.